a United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



?o. I 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. 



CONFIRMATION NO. 



09/716,445 



11/21/2000 



9629 7590 02/1 1/2004 

MORGAN LEWIS & BOCKIUS LLP 
1111 PENNSYLVANIA AVENUE NW 
WASHINGTON, DC 20004 



Hitoshi Okamoto 



046601-5077 



8225 



EXAMINER 



SUJCHAPHADHANA, CHRISTOPHER T 



ART UNIT 



PAPER NUMBER 



2625 

DATE MAILED: 02/1 1/2004 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



w 


Application No. 


w 


Applicant(s) 




Office Action Summarv 


09/716,445 




OKAMOTO, HITOSHI 


Examiner 




Art Unit 






Christopher T. Sukhaphadhana 


2625 





The MAILING DATE of this communication appears on the cover sheet with the correspondence address 



Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)l3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11,453 O.G.213. 

Disposition of Claims 

4) S Claim(s) 1-22 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) S Claim(s) 1-22 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) (3 The drawing(s) filed on 21 November 2000 is/are: a)IS accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)M All b)D Some * c)D None of: 

1 .03 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) (2 Notice of References Cited (PTO-892) 

2) CD Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 
Paper No(s)/Mail Date 3. 



4) O Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) O Notice of Informal Patent Application (PTO-1 52) 

6) D Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 



Office Action Summary 



Part of Paper No./Mail Date 4 




Application/Control Number: 09/7 1 6,445 Page 2 

Art Unit: 2625 

DETAILED ACTION 
Claim Objections 

1 . Claims 2, 17, and 18 are objected to because of the following informalities: In regards 
to claim 2, consider inserting —pieces— after "image data" on lines 3 and 4 of the claim. In 
regards to claims 17 and 18, the claim language seems awkward (e.g. "of each cluster, of the 
image data pieces"). Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

3. Claims 1-3, 6-9, 11, 12, and 16-22 are rejected under 35 U.S.C 103(a) as being 
unpatentable over Yaung (U.S. Patent 6,512,850 B2, newly cited, "Yaung") in combination with 
Desai et al (U.S. Patent 6,072,904, newly cited, "Desai"). 

4. In regards to claim 1, Yaung discloses an image data sorting device (Fig 1) comprising: a 
characteristic value extraction unit (ref no 308, Fig 3, and col 3, line 49) that extracts a 
characteristic value of an image data piece from the image data piece; a relation evaluation unit 
(ref no 316, Fig 3, and col 4, line 58) that evaluates a mutual relation between the characteristic 
values that the characteristic value extraction unit extracts from plural image data pieces; and a 
clustering unit (ref no 318, Fig 3, and col 5, line 3) that executes clustering to events expressed 
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by the characteristic values of the plural image data pieces on the basis of an evaluation result of 
the relation acquired by the relation evaluation unit. 

Yaung does not expressly disclose a sorting register as claimed. However, Yaung does 
teach in col 5, line 21, that image objects corresponding to the image object identifiers in one or 
more clusters may then be easily retrieved and visually displayed on visual output device. Yaung 
does not expressly teach how the image objects in the clusters are displayed or their manner of 
presentation. 

Desai teaches in col 5, line 65, a sorting register unit that sorts the plural image data 
pieces on the basis of a result of clustering by the clustering unit. Specifically, Desai teaches a 
sorted list wherein the identifiers of the images are placed in the list in order of ascending 
distance from the target histogram vector. This ordered list is returned for presentation of the 
images to the user, typically by displaying the first ten closest pictures, then the next 10, etc. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to incorporate Desai 5 s sorting register unit into Yaung 's image data sorting device because it 
would present the user with images in a relevant order. These two teachings are compatible in 
the sense that Desai (col 5, line 63) uses a distance between target histogram vector and data base 
image histogram vectors as a measure of relevance, while Yaung (col 4, line 67) uses a similarity 
value as a measure of relevance. 

5. In regards to claim 2, Yaung further discloses the device wherein: the characteristic 
extraction unit extracts, as the characteristic value of the image data, n types of characteristic 
value vectors (col 3, line 49) expressed by an Ij dimensional vector (col 6, line 20) from the 
image data; and the relation evaluation unit, synthesizing (col 4, line 57) n types of the 




Application/Control Number: 09/7 1 6,445 Page 4 

Art Unit: 2625 

characteristic value vectors as to each of the n types of the characteristic value vectors that the 
characteristic value extraction unit extracts from the plural image data pieces, evaluates the 
relation (col 4, line 64) between the image data pieces in a synthesis result. 

6. In regards to claim 3, Yaung further discloses in col 5, line 3, and Fig 5, "cluster J", the 
device wherein the clustering unit executes clustering by means of a non-hierarchical method 
based on a cluster number. 

7. In regards to claim 6, Yaung further discloses in col 5, line 9, "threshold criteria data" the 
clustering unit executing clustering by means of a crisp technique that does not employ the fuzzy 
theory. 

8. In regards to claim 7, Yaung further discloses in Fig 5, "cluster T\ the clustering unit 
including a cluster number specifying unit that specifies a cluster number when executing 
clustering. 

9. In regards to claim 8, Yaung further discloses the device further comprising: an image 
storage unit (ref no 106, Fig 1) that stores plural image data pieces; and an image management 
unit (ref no 202, Fig 2) that manages the plural image data pieces stored in the image storage unit 
on the basis of a sorting result by the sorting register unit. 

10. In regards to claim 9, Yaung further discloses in col 4, lines 15-26, the image 
management unit managing the plural image data pieces stored in the image storage unit on the 
basis of the sorting result by the sorting register unit and attribute information relating to the 
image data pieces, which are given to the plural image data pieces. 
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11. In regards to claim 11, Yaung further discloses in col 12, line 7, the clustering unit and 
the sorting register unit executing clustering and sorting of the image data pieces stored in the 
image storage unit each time a specific period of time passes. 

12. In regards to claim 12, Yaung further discloses in col 12, line 1, the device wherein when 
there is a new image data piece to be stored in the image storage unit after clustering and sorting 
of the image data pieces stored in the image storage unit, the clustering unit and sorting register 
unit execute sorting of the new image data piece in such a manner that the new image data piece 
belongs to a cluster having the highest relation among existing clusters relating to the image data 
pieces stored in the image storage unit. 

13. In regards to claim 16, Yaung further discloses in col 12, line 4, the device wherein when 
a specific time passes after sorting by the sorting register unit, the clustering unit and the sorting 
register unit destroy the existing sorting and execute clustering and sorting of all of the image 
data pieces stored in the image storage unit. 

14. In regards to claim 17, Yaung further discloses in col 6, line 9, the device further 
comprising an image output unit that outputs image data pieces located near the center of each 
cluster, of the image data pieces after clustering by the clustering unit and sorting by the sorting 
register unit having been executed. 

1 5. In regards to claim 18, Yaung further discloses in col 6, line 9, the device further 
comprising an image output unit that outputs image data pieces having a high relation with 
image data pieces located near the center of each cluster, of the image data pieces after clustering 
by the clustering unit and sorting by the sorting register unit having been executed. 
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16. In regards to claim 19, Yaung further discloses in col 6 5 line 9, the image output unit 
outputs only the image data pieces relating to a designated cluster. 

17. In regards to claim 20, Yaung further discloses in col 11, line 67, the device wherein 
after clustering by the clustering unit and sorting by the sorting register unit have been executed, 
when clustering by the clustering unit and sorting by the sorting register unit are executed again, 
the image output unit again outputs image data pieces after being sorted again. 

18. In regards to claim 21, all the elements set forth in this claim have been addressed in the 
argument of claim 1 . 

19. In regards to claim 22, all the elements set forth in this claim have been addressed in the 
argument of claim 2. 

20. Claim 5 is rejected under 35 U.S. C. 103(a) as being unpatentable over Yaung (U.S. 
Patent 6,512,850 B2, cited above, "Yaung") and Desai et al (U.S. Patent 6,072,904, cited above, 
"Desai") as applied to claim 1 above, in further combination with Lim (U.S. Patent 6,574,378 
Bl, newly cited, "Lim"). 

21. In regards to claim 5, Yaung and Desai do not expressly disclose the clustering unit 
executing clustering by means of a fuzzy technique that employs the fuzzy theory. 

Specifically, Yaung discloses the clustering means using threshold criteria data (col 5, 
line 18). 

Lim teaches a clustering unit (col 7, lines 37-55) executing clustering by means of a 
fuzzy technique that employs the fuzzy theory. 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
to incorporate Lim's clustering unit in place of Yaung's clustering unit because fuzzy clustering 
is unsupervised (Lim, col 7, line 4) and Lim's fuzzy algorithm is the fuzzy equivalent of a 
classical "hard" clustering algorithm (col 7, line 38). 

22. Claims 10 and 15 are rejected under 35 U.S.C. 103(a) as being unpatentable over Yaung 
(U.S. Patent 6,512,850 B2, cited above, "Yaung") and Desai et al (U.S. Patent 6,072,904, cited 
above, "Desai") as applied to claim 8 above, in further combination with Hutcheson et al (U.S. 
Patent 5, 161,204, newly cited, "Hutcheson"). 

23. In regards to claim 10, Yaung and Desai do not expressly disclose the device wherein 
when the image storage unit stores a specific number of image data pieces, the clustering unit 
and the sorting register unit execute clustering and sorting of the image data pieces stored in the 
image storage unit. 

Hutcheson teaches in col 17, lines 23-27, a device wherein when the image storage unit 
stores a specific number of image data pieces, the clustering unit and the sorting register unit 
execute clustering and sorting of the image data pieces stored in the image storage unit. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to incorporate the teachings of Hutcheson into the device of Yaung and Desai because it provides 
sufficiently reliable addressing of element clusters over large databases (Hutcheson, col 17, line 
41). 

24. In regards to claim 15, Yaung and Desai do not expressly disclose the device wherein 
when more than a specific number of image data pieces are added in the image storage unit after 
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sorting by the sorting register unit, the clustering unit and the sorting register unit destroy the 
existing sorting and execute clustering and sorting of all the image data pieces stored in the 
image storage unit. 

Hutcheson teaches in col 17, lines 23-27, the device wherein when more than a specific 
number of image data pieces are added in the image storage unit after sorting by the sorting 
register unit, the clustering unit and the sorting register unit destroy the existing sorting and 
execute clustering and sorting of all the image data pieces stored in the image storage unit. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to incorporate the teachings of Hutcheson into the device of Yaung and Desai because it provides 
sufficiently reliable addressing of element clusters over large databases (Hutcheson, col 17, line 
41). 

25. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over Yaung (U.S. 
Patent 6,512,850 B2, cited above, "Yaung") and Desai et al (U.S. Patent 6,072,904, cited above, 
"Desai") as applied to claim 8 above, in further combination with Sato et al (JP 1 1-136573, 
newly cited, "Sato"). 

26. In regards to claim 13, Yaung and Desai do not expressly disclose the clustering unit and 
sorting register unit executing sorting of the new image data piece on the basis of the relation 
with the characteristic value acquired from the center of gravity of the existing clusters relating 
to the image data pieces stored in the image storage unit. 

Sato teaches in paragraph 0026 the the clustering unit and sorting register unit executing 
sorting of the new image data piece on the basis of the relation with the characteristic value 
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acquired from the center of gravity of the existing clusters relating to the image data pieces 
stored in the image storage unit. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to incorporate Sato's teachings into Yaung and Desai's device because it enables the addition of 
information at one time by collecting the whole classification and enabling it to add information 
by its classification (Sato, paragraph 0014). 

27. Claim 14 is rejected under 35 U.S. C. 103(a) as being unpatentable over Yaung (U.S. 
Patent 6,512,850 B2, cited above, "Yaung") and Desai et al (U.S. Patent 6,072,904, cited above, 
"Desai") as applied to claim 8 above, in further combination with Craver et al (U.S. Patent 
6,233,367 Bl, newly cited, "Craver"). 

In regards to claim 14, Yaung and Desai do not expressly disclose the clustering unit and 
the sorting register unit executing sorting of the new image data piece on the basis of a center 
value of the relations between the characteristic value of the new image data piece and the 
characteristic values of the image data pieces each stored in the image storage unit. 

Craver teaches in col 7, line 67, the clustering unit and the sorting register unit executing 
sorting of the new image data piece on the basis of a center value of the relations between the 
characteristic value of the new image data piece and the characteristic values of the image data 
pieces each stored in the image storage unit. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to incorporate Craver' s teachings into Yaung and Desai's device because the target image can be 
compared with an entire group (col 8, line 2), thereby saving computational time. 
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28. Claims 1, 2, 4, 21, and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Craver et al (U.S. Patent 6,233,367 Bl, cited above, "Craver"). 

29. In regards to claim 1, Craver discloses in one embodiment an image sorting device (Fig 
6) comprising: a characteristic value extraction unit (col 7, lines 1-7) that extracts a characteristic 
value of an image data piece from the image data piece; and a relation evaluation unit (col 7, 
lines 8-13) that evaluates a mutual relation between the characteristic values that the 
characteristic value extraction unit extracts from plural image data pieces. 

Craver discloses in another embodiment a clustering unit (col 7, line 63) that executes 
clustering to events expressed by the characteristic values of the plural image data pieces on the 
basis of an evaluation result of the relation acquired by the relation evaluation unit. 

Craver discloses in a third embodiment a sorting register unit (col 8, lines 37-45) that 
sorts the plural image data pieces on the basis of a result of clustering by the clustering unit. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to combine these the first Craver embodiments with the second Craver embodiment because like 
images would be more likely to be placed next to each other (col 7, line 54). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to combine the first and 
second embodiments with the third embodiment because it allows easy selection of the target 
image from any of the displayed images (col 8, line 59) and images further away from the target 
image is displayed with a smaller size to denote greater dissimilarity (col 8, line 40). 

30. In regards to claim 2, Craver further discloses the device wherein: the characteristic 
extraction unit extracts, as the characteristic value of the image data, n types of characteristic 
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value vectors (col 7, lines 3-7 and 14-16) expressed by an Ij dimensional vector from the image 
data; and the relation evaluation unit (col 7, lines 8-13) , synthesizing n types of the characteristic 
value vectors as to each of the n types of the characteristic value vectors that the characteristic 
value extraction unit extracts from the plural image data pieces, evaluates the relation between 
the image data pieces in a synthesis result. 

31. In regards to claim 4, Craver further discloses in col 7, line 50, the device wherein the 
clustering unit executes clustering by means of a hierarchical method that joins clusters on the 
basis of the relation between the clusters. 

32. In regards to claim 21, all the elements set forth in this claim have been addressed in the 
argument of claim 1 . 

33. In regards to claim 22, all the elements set forth in this claim have been addressed in the 
argument of claim 2. 



34. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. Caid et al (U.S. Patent 6,173,275 Bl) discloses representation and retrieval of images 
using context vectors derived from image information elements. 

35. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Christopher T. Sukhaphadhana whose telephone number is 703- 
306-4148. The examiner can normally be reached on 9a-4p M-F. 



Conclusion 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh M. Mehta can be reached on (703) 308-5246. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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